A Cross-Modal Concept Detection and Caption Prediction Approach in ImageCLEFcaption Track of ImageCLEF 2017

This article describes the participation of the Computer Science Department of Morgan State University, Baltimore, Maryland, USA in the ImageCLEFcaption under ImageCLEF 2017. The purpose of this research and participation is to be able to predict the caption and detect UMLS concepts of an unknown query (test) image by using Cross Modal Retrieval and Clustering techniques. In our approach, for each image (without any caption or concept information) in the test set, we find the closest matching image in the training set by applying similarity search (e.g., content based image retrieval) in a combined feature space of color, texture, and edge-related visual features. By linking the associated caption and UMLS concepts of the closest matched image, further processing are performed to extract terms (keywords/concepts) to form a text feature vector and finally return the top ranked terms as predicted concepts (caption) from the best matching cluster centroids which are previously generated by applying K-means clustering in a term-document matrix of the training set. In this article we present main objectives of experiments, overview of these approaches, resources employed, and describe our submitted runs and results with conclusions and future directions.